Section: Software and Platforms

The Substance Middleware

Participants : Michel Beaudouin-Lafon [correspondant] , Clemens Klokmose, Tony Gjerlufsen, James Eagan, Clement Pillias.

Substance is a middleware based on a novel programming paradigm called data-oriented programming and was designed to facilitate the development of multi-surface interactive applications  [45] . Such applications are distributed by nature as they involve a varying number of display and interaction surfaces that are controlled by different computers. For example, our WILD room includes a 32-monitor display wall driven by 16 computers plus a front-end, a multi-touch table, various mobile devices such as iPodTouch and iPads, and the laptops that the users of the room may bring with them. We want to support seamless interaction techniques across these surfaces, such as the pick-and-drop technique pioneered by Rekimoto  [47] .

Data-oriented programming consists of attaching functionality to a tree data structure through facets attached to the individual nodes of the tree. Facets can be added and removed dynamically, and notified of changes in the tree. Substance supports two powerful ways to share nodes and facets: mounting, where access to the shared tree is managed through remotely, and replication, where the shared tree is replicated at each site and synchronized.

Substance has been used to create two full-scale applications (Figure 7 ): a generalized Canvas that can display and manage graphics, PDF files, image files and other content (through an extensible content manager) across surfaces spanning multiple displays and computers; SubstanceGrise, which uses multiple instances of the Anatomist/BrainVISA application to display coordinated 3D imagery of many brains in parallel on the WILD wall and control from a physical model of the brain.

Substance is available at http://substance-env.sourceforge.net/ under a GNU GPL 3.0 licence.

• ACM: H.5.2 [User Interfaces]: Graphical user interfaces (GUI)

• Software benefit: See T. Gjerlufsen, C. Klokmose, J. Eagan, C. Pillias and M. Beaudouin-Lafon (2011) Shared Substance: Developing Flexible Multi-Surface Applications. In CHI '11: Proceedings of the 29th international conference on Human factors in computing systems. ACM, pages 3383-3392.

• OS/Middleware: Mac OS X, Linux

• Required library or software: several, managed by Python install

• Programming language: Python